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Abstract 

We studied the influence of mechanical vibrotactile signals in the acoustic 
range to the visual perception of flickering images. These images are shown 
on a CRT screen intermittent at about 75 Hz, without external perturbations 
are perceived as constant and stable. However, if presented together with a 
controlled acoustical vibration an illusion is perceived. The images appears 
to float out of the screen, while the rest of the room is still perceived normally. 
The acoustical signal given to the subjects were of very low frequency (below 
100Hz) and low amplitude (almost inaudible). The stimuli were transmitted 
through direct contact to the subject's chin with the use of a plastic stick 
connected to a speaker. The nature of the illusion is described and a basic 
theoretical model is given. 

Keywords: Illusion, vibrotactile, blinking, flickering, visual stimuli, visual 
perception, acoustical stimuli, image formation 
2000 MSC: 91E30, 91E45 



*Tclcphonc +81.45.787.2167 
Email addresses: ruggero@yokohama-cu.ac.jp (Ruggero Micheletto), 
maria-o_o@i . softbank. jp (Maria Fernanda Avila Ortega) 

1 First names: "Maria Fernanda", telephone +81.80.4090.5457 



Preprint submitted to Elsevier 



November 28, 2012 



Vibrations are everywhere. When we walk, all our body shakes. Our eyes 
are not exception, they jitter while we move. As a consequence, images pro- 
jected on our retina are trembling, moving chaotically in random directions 
(Murakami and Cavanagh (1998)). 

Moreover, we are continuously scanning what we are looking at, in jerky 
criss-cross movements that are called saccades. So, even in the absence of 
external mechanical perturbations, the images on the retina are randomly 
shifting because of the saccades. Despite of this, the brain is able to remove 
spurious movements and produce the perception of clean and stable images 
in any condition (Vercher et al. (1984)). 

This "cleaning up" elaboration process is a heavy and complex task, how- 
ever we are not aware of what is happening, nor we feel any effort. It is a 
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In this study we perturb this process to make it surface to the conscious 
level in order to investigate its characteristics. We describe a simple experi- 
ment that, interfering with the the visual elaboration process, induce errors 
that can be perceived at the cognitive level. The method combine the in- 
teraction of an acoustic vibrotactile signal and the perception of flickering 
images. 

To generate the flickering, we used a cathodic ray tube monitor (CRT). 
This once common device draws images point by point with an electronic 
brush that scan the screen at a frequency of 75Hz. 

We setup the following apparatus: a controlled acoustical vibration of 
about same frequency as the display is transferred to the subject eye by a 
small wooden stick glued to a round plastic cylinder that is placed horizon- 
tally under the chin as shown in figure [TJ The stick is mechanically connected 
to a speaker that the subject simply holds in his hand. Because of its low 
amplitude and frequency the vibration is barely audible and does not disturb 
the subject, however the sound is efficiently transmitted to the whole head. 

The frequency is generated by a sinusoidal function generator while the 
subject is seated comfortably looking at a geometric image on the screen (see 
the image in figure [2^,) . 

While varying the frequency and amplitude of the sound, we asked the 
subjects to say when the perception was becoming unstable. 

When the audio signal reaches a frequency close to that of the screen 
flickering, something interesting happens. While the rest of the room ap- 
pears unchanged to the subject, the images presented on the CRT screen 
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Figure 1: The very simple way we applied the acoustical perturbation to the subject retina. 
The vibration is in the 50^100 Hz range and of very low intensity (almost inaudible) 



begin to move and appear unstable. The images perceived are distorted with 
perturbations that change in strength and shape in line with the audio signal 
intensity and frequency (see figure^). 

We asked the subjects to report 4 levels of intensity to quantify this 
subjective phenomenon: no illusion (image perfectly stable), weak, medium 
and maximum. In figure ^ are plotted the frequencies of maximum effect for 
different subjects. The illusion appears and decays with a Gaussian profile 
as shown in the plot Q. 

It is worth repeating that all other objects in the room are perceived as 
perfectly still, only the images on the display appears to oscillate (a schematic 
representation of the percept is in figure^). 

What is the cause of the illusion? If the retinal persistence is the only 
reason why we perceive stable images from the flickering stimuli, even the 
vibration introduced by the external acoustical disturbance, that have similar 
frequency, would average out and have no effect, but this is not the case. 

In other words we cannot assign the illusion to the sole effect of the 
mechanical vibration that perturbs the retina, because all the other objects 
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Figure 2: Panel a). The image presented to the subjects. Panel b). A representation of 
the perceived illusion: a wavy distortion of the picture that appears also to dynamically 
float off screen 



surrounding the display are perceived normally. It the interaction of the two 
causes (flickering and vibrotactile signal) that give rise to the illusion. 

To have an insight about this, we use a simplified treatment. We define 
the image on the retina as a two-dimensional map 

M = P x y(t) 

and the perceived pattern: 

where p represents the intensity of the point and the xy apex indicates that 
the parameter is a bidimensional map in the xy coordinates (from now-on the 
tilde always indicates perceptions). We then represent the external acoustic 
signal disturbance as perturbation of pulsation u , in this formalism the 
general expression relating patterns to percept is : 

M = T W °[M}. (1) 

The function T is the unknown transfer function that represents the com- 
plex algorithm that the brain uses to generate the perception from pattern 
M. The apex w represents its dependence on the external vibrotactile signal 
frequency. 

Clearly, we do not know the exact form of T, however we can derive rel- 
evant characteristics of it from the illusion. We consider two optical stimuli, 
the fix objects external to the CRT screen 

M e = p* V 
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Figure 3: The frequencies that resulted in the maximum illusion strength for several 
subject. Some subjects performed the experiments twice or three times, for them the 
error bar is given 



and the nickering images presented by the screen (for simplicity we assume 
all screen pixels blink simultaneously) 

M s = ps V cos(u s t) 

the indexes e and s represent the external and screen stimuli respectively. 
When we perturb the visual system with an acoustic frequency u , if the 
stimulus on the screen have a different pulsation u s , the visual system has 
the ability to average everything out. So T have this form: 

M = T^ [ Ps xy cos(u; s t)} = p* y , 

the percept is a constant pattern as seen experimentally before 60Hz or after 
85Hz in figure |4j However, when the two pulsations are comparable, the 
illusion prevail and subjects perceive fluctuating images (peak in figure [1]). 
The percept is not stable, oscillating with pulsation Q: 

M = p xv cos(fit), 

we can write: 

M = [ P s y cos(w a *)] = Ps y cos{ttt) 
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Figure 4: The strength of the illusion is measured by a subject in terms of four subjective 
levels: maximum, medium, weak and no illusion. To plot the data a value is assigned to 
each of the level as following: maximum=3, medium=2, weak=l and zero for no illusion. 
The effect appears when the acoustical perturbation is comparable with the flickering 
frequency and decrease when it exceed it with a Gaussian profile as shown. Blocks are the 
average values for subject RM and OK and continuous curve is the fit. 

naturally Q depends on the frequency of the external vibrotactile pertur- 
bation. It is a function of w different from zero only when the acoustical 
perturbation and the flickering stimulus frequencies are similar. The shape 
of Q is in fact deducible from experiments and coincide with the profile of 
the curve of fig. |4j This can be modeled with a Gaussian function as: 

n = {u)-u )e-( w - u °W S ', 

where 5 is a parameter proportional to the spread of the curve. This value 
can be estimated from out data as 5 = 10Hz. 

We studied systematically the distance dependence and found that this 
visual effect tends to be more prominent ad distance and to decay in close 
proximity of the screen. Data in figure [5] show that at very close distances 
the illusion is not perceived. At these ranges, image covers a wider area on 
the retina, the number of receptors involved is high enough for the brain to 
process the image avoiding any confusion. On the other hand, the illusion 
gets more dominant at distance for the opposite reason: the image on the 
retina gets smaller, involving less cells; the brain has less information to 
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Figure 5: The strength of the illusion in function of distance. The screen was placed at 
increasing distances and the subjects were asked to give a relative evaluation of the illusion 
strength in percentage. TOT symbol represent the average of all subjects. 



process and the flickering disturbance has an easier task to prevail. We would 
expect a monotone increase in the illusion strength at greater distances, 
however as seen in figure |5j some subject exhibit a decay due to intervening 
visual resolution problems. 

To have a confirmation of the phenomenon, we modelled the the eye as a 
perfect lens, with the retina placed at focal distance length L=17mm. Con- 
sidering that the pattern stimulus is about P=20cm in size, simple triangu- 
lations allow us to calculate the size of the image on the retina as h r = L * ^ 
where d is the distance of the stimulus from the subject eye. Supposing that 
the perception occurs within the fovea and taking in account that typical 
cone density is of about 150.000 cones/mm 2 , we plot the number of recep- 
tors involved against the distance. These data compared with the illusion 
strength shows that when the number of receptors drops, the illusion start to 
kick in suggesting that the brain algorithm robustness depend on how many 
receptors are involved in the elaboration, see figure [6j 

If the reader is curious to test this optical illusion by himself, he does 
not need a complicated setup. He just needs to find an old cathodic TV 
or monitor, switch it on and place himself at about 2-3 meters from it. To 
perceive the floating display illusion, it is not necessary a calibrated per- 
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Figure 6: The strength of the illusion in function of the number of receptors involved. As 
the number of receptors drops, the Brain loose its ability to stabilize the flickering image 
giving rise to the illusion. 



turbation of the right amplitude and frequency as done in this study. The 
easiest way to perceive the illusion is simply eating a cracker or a biscuit few 
meter away from the blinking image. Crunching the biscuit will stimulate a 
broad spectrum of frequencies, also those in the correct range, giving rise to 
the effect. You will notice that the images on the TV appear to float out of 
the screen, while all the rest of the room appears unaffected. We are sure 
that this optical illusion has been noticed by most people with a sufficient 
level of attention in casual moments of everyday life. 

The experiments presented show that the algorithm used by the visual 
system to stabilize images from saccades and external vibrations, can be dis- 
rupted and fail in presence of blinking images. Using a controlled vibrotactile 
mechanical stimulus on the subject, we found that this happens at frequency 
ranges similar to the one of the image, even if it is not perceived as blinking in 
normal conditions. Our study shows also that the phenomenon is dominant 
at longer distances when solid angle is smaller and less visual receptors are 
involved. We give a simple theoretical model that finds the explicit Gaussian 
relation between image blinking frequency and the vibrotactile stimuli. We 
believe that further study of this effect may lead to a better understanding of 
the relation between retina vibration stimuli and visual perception, shed light 
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on the mechanism related to the dynamic formation of images and maybe 
help the development of methodology to discover visual perception related 
diseases. 
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